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ABSTRACT . • 

The preparation of this simulation material p 
is guided by the concept of an evaluator as a decision-maker, 
on the definition of evaluation as a continuous assessment co 
with answering decision-making questions. The continuous cone 
©valuation is based on the model created by Egon 'Cuba and Dan 
Stuffiebeam, named by its acronym CIPP — context, input, proce 
product evaluation. Sections are devoted to describe and expl 
CIPP evaluation model. The intent of this evaluation game is 
provide an instructional tool for applying the concept of con 
evaluation to a reading program. Its specific goal is to teac 
participant to use one evaluation model as a guide to knowing 
do and what-^to decide. The matefials describe a school system 
hires an evaluator who must help conceive -a reading program a 
as a"ss"ess it. The incidents, described try to simulate actual 
situations. There are three kinds of pages in this simulation 
each marked by a different color; white pages carry descript' 
situations, green pages are response sheets, and, pink pag 
•'omniscient comments" or answers proposed by the authors. 
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Definition of EvaLu:ation 

In the last two or three decades evaluation ha.^ been a changing concept. 
The term has been used to encompass a variety of activities: conducting 

experimental research -grading people, judging subject areas ty general 
lists of criteria, comparing scores from standardized tests, even giving 
"gut" reactions to programs under consideration. 

As evaluation is defined in this project, however, it is disciplined 
system for helping decidion-makers : evaluation is a continuous assessment 

concerned .with answering decision-making questions. For example, at the 
beginning of a project a decision-making question is: "tfhat should this 
Title I project focus on?" At the end of the project another decision-making 
question has to be asked: '*Is this reading program worth continuing next 

year?’*' And, of course, there are many other decision-making questions 

during the course of ^e project. - 

Several educetors heve etten^ted'to describe contf.neous eveluatlon using 

different diagraos and temlnology. Michael Scrlven speaks of fomnatlve 

•• *■ da«lop»; coica £o»rd) 

■ and suimaatlve evaluation (evaluating a program when everything is finished), 
in other words.' how does it all add up7, Robert Stake talks ebo.nt. entecadents 
(What exists prior to, the project?), intentions (What are tha goala of the 
projadet). and tranaactlons and outcomes (What has happened as the project 

hss developed that will affect the results?). 

The continuous concep^: used in the Simulation Training Package, however, 
is the. evaluation model created by Egon Cuba and Daniel Stuff lebeam. Their 
mode l is named by its acronym CIPP — context, input, process, and product 
evaluation. The'pl^^sisl^adic^'a constant con« 
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from the ioicial stages of determining needs to the concluding stages of 
examining the final product. 

Continuous evaluation includes several basic activities: creating a 

proper climate for evaluation, training personnel .to become sensitive to 
decision-making stimuli, collecting data for feedback, modifying operations, 
and emphasizing decision-making for all project personnel. 

Part of the evaluator’s responsibility is to create the proper climate 
for evaluation. Therefore, the. evaluator ia not a d^a collection technician 
but rather a manager -with plans to carry on evaluation throughout the course 
of the project. Additional personnel must be trained, however. Evaluators, 
most of whom have other responsibilities, cannot possibly conduct all of the 
evaluation activity personally, so they muot train other people to help. But, 
these added personnel imjst be oriented and made sensitive to decision-making 
stlmuili. For example, a superintendent or a principal will be ptesaured 
by Pi T. A. groups, community action groups, or student groups to install 
additional equipment or to alter a curriculum:. An evaltwtor should be 
sensitive to these pres8ur«as try f§e them gp that he has some 

plan, some program ready to meet the problems associated with pressures on 
the decision-maker. 

Collecting data to. feed back into the progtas® ito constantly modify the 
operations ter^other phas.a-of continuous evalua^lom. Decisions mast be 
made often if a program" is to remain up-dated. 

In developing a curriculum . there are many levels of decioio'ffi-making as 
well as many different roles for declsioa-makerB., Ftot the federal level 
to the local level, decisions ere made about the; value of projects. For , 
eacannple^ the U. S. Office of Education must deciiae whether or not its Title I 
money is bnylng services that are improving the rreading of disadvantaged 



students. Also, the Office of Education must collect data about their programs 
before asking Congress to continue them. On the local level, too, a superin- 
tendent must decide whether or not the project wuld benefit his system. And, 
finally, a teacher decides if project data indicate benefits to her class. 

An evaluator, therefore, cannot collect all of this data himself, so he must 
train and involve these personnel in collecting information that will enable 
them to uiake accurate decisions. 

Hhat deterolnes how a decision is medeT tf the definition of evaluation 
IB focueing, delineating; and collecting data for aa^erlng deciston-maklng 
ouestlons, tnen the criteria for evaluation will come from each decision to 
be made. Criteria such as relevancy, timeliness, economy, and feasibility 
will be applied to a project to determine how it wiki proceed. 

Individualized criteria for each project require that data be collected 
by measpres other than standardized reading tests and standardized > 

intelligence tests. The hard data from tests still have co te collected, 
but other information is also needed to answer decision-making questiona — 
questionnaires, interviews, attitude inventories, and so on. Therefore, the 
evaluator’s plan Wt identify the kinds of data that will be needed, the way 
they will be collected, the people. whb will collect them, the best way of using 
the data to answer the questions, and the means of interpreting them. 

TWO types of decisions made' in continuous evaluation arc congruency 
decisions and contingency. decisions. Congruency decisions determine whether 
or not products and outcomes match objectives. For example, a teacher 
requesting students to read five books outside of class during the course of 
a semester must also examine' his procedures tb discover if students ’are reading 
the books, if the 'available materials are being’ used, and If more materials 
should be ptur chased* 
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Contingency decisions are concerned with determining the effectiveness 

of fisting operations. p6ssit>le situations requiring contingency decisions 
would be these: Does an inf priori reading inventory enable a teacher to 

adequately diagnose disadvantaged children? If a crisis occurs, the directors 
of a project must decide whether or not their program should be modified. 

Those are contingency decision^ « 

/ Vno tfier aspect of the conttnuoiss evaluation definition, which is not 
Innsediately apparent, is that educators must be accountable for the dollars 
and the tisie spent on educational programs. Accountability should not be 
equated with money, even though money is where emphrsis is often placed. In 
the past, evaluation^ reports were deaertptions. The current definition of 
evaluation, howevpr, pictures the evaluator not only as a iTescriber but also 
as an interpreter of events th^t will help decision-makers make their 
assessments^ Therefore, the evalustor must have plans for selecting and 
interpreting data, cs well as for providing information in a usable form 
for the decision-makers - - the c,conPUjner8 of that information. 

7 ^. . 

Role ^ Che Evaluator 

What implications for the role of an evaluator are inherent in the 



continuous evaluation concept? His work is not finding answers to problems; 
finding answers is the goal of evaluation. He is neither a data collection 
technician, nor is he limited to merely producing descriptive data. Rather, 
the role of an evaluator embraces dual responsibilities as manager and as 





judge,' An evaluator is a manager of data; he specifies objectives, monitors 

'f ' 

processes, collects data, assesses and Interprets data, and writes reports. 
Necessarily, he- must create ai* evaluation climate and then ask questions that 



win focus the decision-maker* a eyas on apprprlate actions that accomplish 



-stated goals. 
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In terms of a role then, the evaluator is not an adjunct, a barnacle on 
the side of the decision-making vessel. He is rather a participator in the = 
decision-making process. Moreover, the evaluator may even develop the 
managerial plan for a project - identifying the objectives and goals as well 
as the contingency and congruency decision points. He brings information to 
the decision-maker and helps him disseminate it, if dissemination beyond the 
deeisicn-maker is needed. As soon as feedback to the consumer has been 
accomplished, the evaluator's job, in a sense, is complete. However, feedback 
to the decision-maker means feedback into the program and that recycling 



contlnuslly involves the eveiuatoir# 

The concepts ot a pertlclpatlng evaluator and of continuing evalustlon 
guided the preperntton of the meterlala In the Slanilation Training Packege. 






9 







o 

ERIC 



0 



at*' 



8 






A Concept for Evaluation 



A person who has the responsibility for evaluating a xeadlng progra^a 
fac^s the task of determining what program he will examine as well as 
Vhat: decn he will collect in order, to make a judgment about the value of 
a pfograai/ Several educators are now theorizing about appropriate strategies 
for evaluation. TbV are proposing a number of concepts or models in which 
the practicing evaluator can operate. The bibliography identifies several 
the'^mtical models for evaluation, W these training materials use a 
aod^l developed by Egon Cuba and Daniel Stuf flebeam as the framework in 
which it 'discusses the evdWtion of reading programs. This model was 
chpSeh for two reasons it is simple and it has suffered through the 
revlsloA of time and trial. 

The model that appears in these materials is called the pff? Model , 
au Acrudym standing fcr Contest , Input , Process, Products ^It is based 
upoft the assumption that educational, programs should be evaluated on a con- 
tihuous basis, from start to finish, rather than on the final product alone. 
Itx other words.^an evaluator should be part of the team that looks at the 
tiuede'of the school and determines the kind of program the school needs to 
teaoh treading more effectively. And he should continue to be involved with 
the pfogrard throughout its movement, toward the end-product. Context, input, 
prccaes, and producit refer to various stages of a program*'* movement 
Completion. 

Context evaluation Is concerned with the setting or the envlroment 
ivn Which education Is taking place. It includes the community, the 
children, the teachers, the buildings, and the present approaches used to 
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teach reading. The basic question that has to be answered undor the 
heading of context evaluation is, "What are the needs that should be 
satisfied? " 

Input evaluation translates the decision made under context to 
specific objectives and procedures for achieving the' general objective 
implied In the 'Context decision. The ans^rs to, "What are the best 
means to solve tlie need or the problem?" provide the program plan and its 
procedures. 

Prjpcess evaluation Is an examination of th^^rogram as it functions 

\ 

to see if the pro^am plan meets expectations or If modifications should 
be made In operatlnns. This kind of evaluation not only wants to see 
whether everything works or not, but also wants to provide an avenue for. 
making modifications where the need indicates." 

Product evaluation analyzes thevresults to see if the objectives of 
the project were. .met. Some evaluators refer to this kind of assessment as 

r , . 

summatlye evaluation. T3n>ically, product or summatlve evaluation is the 
only. kind of formal evaluation provided in program assessment. 
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Reading and the CIPP Model 

It Is not enough merely to point out to evaluators a conceptual 
framework within which to -operate. People must first develop an attitude 
that encourages high 'performance -'in a glven<ja^ea and identify and become 
skilled in the tools of that subject. ..Therefore, the objective of these 

' y ' 

y ' ■- 

materials goea beyond Identifying the CIPF Model as a means for conceptualizing 



evaluation. It Is necessary in addition, to. see how reading programs ‘ fitt 
into that framework and to learn how to use the tools that will, effectl^ly 
collect data* on reading programs. 
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What are the variables in a reading program that an evalutor wants 
to concern himself with? How can he Identify needs related to the reading 
program, especially a reading program for disadvantaged youngsters? What 
are appropriate behaviors that can be expected of youngsters? What termi- 
nology has to be used to express those behaviors for reading teachers and 
for administrators? What are the major decision-making questions that 

r" 

have to be answered about reading in terms of objectives, procedures, and 
final performance? What instxruments and techniques can be used to monitor \ 
the reading process and the teaching techniques that attempt to produce 
adequate reading behavior? What background and skills should a reading 
teacher be expected to have? How does one assess the reading performance 
of the students? These specific questions make it essential for evaluators 
and reading specialists to work together. These training materials help 
to identify ways in which the evaluator and other specialists contribute 
to the evaluation of readings , 
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THE CIPP EVALUATION MODEL* 



9 



o 

to 

c 

(d 



§ 

O 



OS 

u 

•o 

00 

c: 



to 

9 

rH 

OS 

> 

U 

M 

O 

ca 

0) 

•H 

00 

0) 

4-1 

OS 

M 

4-1 

CO 

o 



0) 

*c: 

o 

CO 

c: 

o 



m 

o 



CO 

CO 

os 

<< 



0 

M 

1 

I 



M 00 

O O 



0 -U 

01 9 

4J g, 

00 0 

«r4 

00 00 
01 00 o 

00 f-< p -u 
CO ^ W 4 
0) OS -H P 
CO f-l CO 4-t 
00 -*4 0) 00 
OS OS *o 

> o 



to x6 

a ^ 

•H M 

N O 

>» ti 
f-l os 

g e 

fS *0 
0 

'O OS 
fl 



d 


d 44 


60 § 


0 fk 


0 


0 


60 


d d 


>% 0 


- d 


i4 4S 


4-r i4 


0 i4 


.o 


i4 44 


0 44 


1^ 0 


44 1-1 


-H d 


U r-4 


d r-4 


^ 0 


o .o 


0 1-1 


0 B . 


0 0 


*0 43 


44 0 


0 f-4 


i4 0 


0 f-4 


•O iH 


Of 
O 0 


a !• 


0 

> 


H U 


0 -H 

0 

44 

0 


P6 0 


O 


0 


1 


44 


d 


^ r® 




iH 


d d 


ff 0 


f-4 


1-1 0 



ca 

01 

•H 

00 

O’ M 
4J O 
OS M4 
M 

44 00 

s 

a -H 

o ca 

•H 0) 

44 'O 

ss 

1-4 1-4 
O CO 

00 u 

0 

-.•o 

ca 01 

01 u 

S 8 

a 

o 

00 'O 

01 0 

M CO 



d 

o 



u 

•O oS 
d 
os 

o 

?s 

44 0) 

•H 00 
r-4 M 

2 

JD O 
•H O 
CD 

OS 0 

01 Xi 

CM 44 . 

d 

d 0) 

0) -r4 ^ 
d OS 
C ^ 44 
OS B 
> Q 0 

0 cs .o 

f-4 o 

01 o o 

h 0) 44 



^ CO 

44~H— 

M 0 

O 

Q, 01 60 
pu u d 
SS O -H 
00 »4 0 

p. OS 
CM M 

O *0 S) 

d o 

0 os w 

01 cu 

O 0 

M 0) M «- 
d -H O « 
O t0M4 01 
0 0) -H 



g> 



OS * 
M 0) 



44 0 t4 4J 

O ^ O 
0) d os 
«-4 o 03 

0) -H d O) 
SO 44 M OO 

0 -H a 

M,f-4 0 0 

O O 0)4C 
0 *0 O 



00 

d 

•H 

0 >^ >^ 
^ a 

f O d 
d 0) 2 
O 4C d 
*H H d 
0 ►O 
•H 44 

a 0 ^ 

- -44 

d 

0 0 
Vi 0 0 
O 0) & 
M-l 0 O 
0 t-4 
44 << 01 

d ^ > 
d d ^ 



& g 






•H •*= 

X 44 

ti ^ 

0 -H 

O 

O 0 

d 0) 
O 0) 
•H d 



0) 

*o • 
0 

•O 'O 

d o> 

os d 

4-1 0) 

•H 4C 



•O 

d 

os 



p« 



44 




d 


0 


0 


0 60 


M 


0 


*o d 


0 


0 


iH iH 


Of 


0 


Ps 


o 


tt 


O r-4 




0 


44 S4 



0 


0 


X *o 


TS *0 


44 d 


d d 


0 

0 


0 0 


d 


M m 


iH CM 


4-) D 


CM iH 


X 0 


0 44 


0 f-4 


*o d 


ti rO 


0 


d o 


o ts 


O t-i 


H i4 


C4 Of 



>» B >> 

f-4 43 

r-4 O 

§ «jd • »' 

44 44 

•O X 

•H 0) 0) 

> > ti 

t 4 -H d 
-O 44 O 

duo 

•M 0>^ 

Of 0) 
0^ 
>4 44 
•M 0) 

•O Of«M 
Tl O 

U 44 

Odd 
0 0 0 
0 ) > 0 > 
*0 0 44 

>» 0) >> 
OQ M 0 



g> 



D ■ 
Of bO 

44 d 

d *r4 

o M 

►O »-4 

d 0 
0 c 
0 

0 

44 

d .o 

Of 

d *0 
•H d 
0 
•o 

*§ § 
0 0 
4J 44 

d 0 

•H >* 
0 

-o «o 
.d d 
0 0 

r-4 0 

0 x: 

0 44 
44 

O (M 
0 O 



0 

d 

0 O 
0 1-1 
1-1 44 

2 ^ 

d 0 

0 44 
Of d 
0 1-1 
Ki 

o *o 
0 d 
1H 0 

•o 

o 

MH 0 
O -H 

44 

0 1^ 
0 f-4 
0 0 
0 0 
0 44 

a o 

0 

0 ^ 
f-4 d 

•O 0 
1-1 0 
0 ^ 
0 44 
O 0 
Of 43 



0 

^ fC 
O 44 
44 1-1 

1-1 0 
44 44 
44 0 
.0 «r4 
0 O 
O 

0 0 

•d 0 

44 Xd 

d 0 

o *^4 

Of 0 

d o 

60 

60 

0 0 
1-1 4C 

*0 44 



0 

0 0 0 
0 8^ 
> 0 d 

-r4 f-4 0 
44 rd 

0 o a 
0 u 
•r-l.Of 
.O 0 

O 60*0 
d 0 
0 1-10 
fd > d 
44 f-4 

O 60 
*o 0 d 
vd -n 
0 X d 

44 d 
0 1-10 
•a > f-4 

0 Of 

0 *o 
d 0 M 
44 o 



0 «f 60 0 


to 


0 *o d -H 


0 1 


*0 0 «H u 


O 1 


> 44 O • 


b 1 


(4 W 0 0 0 




O 0 0 0 « 




pM 0 8 0 -H 


CO ' 



0 0 
44 o 

Xi 0 
60 0 
1-1 u 

f-4 0 

d <44 
bq d 
o 

0 u 
0 

60 
d d 

O 1-1 
1-1 ^ 
44 U 

gg 

0 0. 
> JC 
W 44 

44 

S ^ 
0 *0 
0 0 
fO U 
0 0 
f-4 > 

<M f-4 

d 0 
44 *0 
C/3l 
I 0 
- 0 
•-4 . 0 

r-4 *0 



i 

f-4 

d 

u 



*o 

d 

0 

d 

o 



0 

1-1 

d 

s 



0 • 

Of 00 

d ^o 

CO 

f-4 

>4 

o •* 

<M « 

o 

d 

1-1 o 

44 

0 

i4 d 

o o 

O 44 
0 60 
0 d 
<3 v4 
fd 

0 0 
xi 0 
E-< :s 



pq 

E-t 

M 

S 

o 







o 

ERIC 






THE CIPP EVALUATION MODEL (continued) 
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TABLE B 

developing evaluating designs outline* 

The logical structure of evaluation design is the same for all type 
of evaluation, whether context, input, process, or product evaluation. 



A. 



B. 



Focusing the Evaluation 

1, Identify the major level(s) of decision-making to be served, 

e.g., local, state, or national. , ^ , 

2. For each level of decision-making, project the decision 

situations to be sc and describe each one in te^ of 

its locus, focus, cri-ti.cftMty, timing, and composition of 

alternatives. . . 

3 Define criteria for ceseh dseclstea situation by specifying 

variables for measurement 2 md {S^ndards for use iin thei ju gsnent 

of alternatives. 

4. Define policies within’ tbes eva lust .^on must operate. 

Collection of Information ,, 

■r Specify the source of f;he £nfor=M±ion to be collected. 

2. Specify the instruments.' esad: met&»*ds for collecting e nee -e 

information. 

3. Specify the sampling prmcsacure feo be employed. 

4. Specify the conditions Amri scheamle for information collection. 

Organization of Information ... . 

1 . Provide a format for the information which is to be collected. 

2. Designate a means for coding, organizing, storing, and 
retrieving information. 



D. Analysis of Information 

1. Select the analytical procedures to be employed. 

2. Designate a means for performing the analysis. 

E. Reporting of Information 

-1. Define the audiences for the evaluation x'eports. 

2. Specify means for providing information to the audiences. 

3. Specify the format for evaluation reports and/or reporting 
sessions. 

4. Schedule the reporting of information. 



F. Administration of the Evaluation 

1. Summarize the evaluation schedule. 

2. Define staff and resource requirements and plans for meeting 

these requirements. . c 

3. Specify means for meeting policy requirements for conduct or 

the evaluation. ,, ..jj 

4. Evaluate the potential of the evaluation design for providing 
information which is valid, reliable, credible, timely, and 

pervasive. i: u 

5 • Specify and schedule means for periodic updating of the 

evaluation design* 

6* Provide a budget for total evaluation program. 
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*Daniel L- StufflebeamT "Evalluat-iam as Enlightenment for Decision-Making," 
p. 42. 
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TABLE C 



CIPP MODEL 



evaluation 

Identify a problem (unmet need) . 

Focus the problem. 

Determine criteria for problem satectlon. 

Collect data on problem areas. ^ 

Determine system (organization^ rasillities, materials, 
schedules, teachers, community, etc;.) capabilities. 

Select specific problem based on at&ove criteria. 

Input evaluation 

1. DeterotLne broad objectives or goal^ for any possible 
alternatives for meeting problem. 

2. Specify criteria for program alternative selection. 

3. Gather and/or develop possible program alternatives. 

4. Assess alternatives based oh criteria specified. 

5. Select alternatives and refine objectives and program 
description. . 

Process evaluation 

Develop plan for constantly monitoring program. Plan 
should be contingent on — • 

a. Decislon-tulcer 

b. Decision to be made 

c. Type of decision to be made 
. d. Timing of decision 

e. I^ta collection capabilities 

f . Flexibility possible within program 
Assist staff in focusing problems as they arise; assess . 
alternative strategies for meeting problem and monitor 
effectiveness of alternatives selected. 

Implement monitoring plan. 

Product evaluation 

1. Develop plan for collecting data on program goals and objectives 
(Different levels of authority are Interested in different 
objectives and goals.) Plan should be contingent on -- 

a. Decision-maker 

b. Decision to be made 

c. Type of decision to be made 

d. Timing of decision 

e. Data collection capabilities. 



2 . 

3. 



Context 

1 . 

2 . 

3. 

4. 

5. 

6 . 
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Directions 

How to Use the Simulation Training Materials 

The pages which follow are mailed simulation training materf^ls- They 
describe a school system that hires an evaluator who must help emneeive a 
reading program as well as assesis it. The incidents in these pages try to 
pictsire (simulate) the kinds of things that occur in actual sitiistions. 

In order to achieve the intent of the materials, the particiapant should 
"play the game." And this is indeed a game for educators interested in 
improvin^programs through better decision-making processes. But playing 
the game necessarily limits freedom of response. Evidently the limits of 
a book prevent the participant from following to a conclusion some game- 
declsions that he makes, even though they may be better decisions than the 
ones proposed in the authors' gam^ plan. Thus, make hypothetical decisions 
■in these pages, but accept the decision that the authors give in their 
'’omniscient comment,'* 

The intent of this evaluation game is to provide an Instructional tool, 
a vehicle, for applying the concept of continuous evaluation to an educational 
program, a reading program. It has the more specific intent of teaching the 
participant to use one evaluation model as a guide to knowing what to .do and 
what to decide. Again — the authors understand, as you do, that there are 
several models that could be used. The CIPP model has been selected as a 
sample model, and hopefiully a sample model that is complete and that provides 
a series of clear steps to guide the evaluator throughout. 




Format 

There are three kinds of pages in this simulation game, each marked by 
a different color. White pages carry the description of how things are 
progressing in Git, Mo. , Unified School District. White is real life. Green 

IG 



pages are ^'Response SbeeCs" where you react or mak- * dectsioa In the real- 
life world of Giz. Pick pages carry the "CtamiacleP^; Ctoimcaits'' or the 

answers • 

The response sheets (green) shornld be used at . ?esk scratch pads. Iteke 
notes about what a situation is and how a problrem c.io Iwe bandied. Do not 
look at the ’'oouisclent coniients" (pink) until you i have made your response 
on the green sheets. Though the pink sheets are cajlled •’•©nmlscient Comments 
they are actually only plausible alternative aolutriwna — not necessarily 
the best. But they are the decisions and the solcitiions (that the participant 
has to accept in order to proceed and to continue '’•play the game 



It 
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CLARIFICATIONS AND DEFINITIONS 

a) The term Individualized readloR is used here to' mean any method 
that will provide more attention to the needs of individuals. 

b) This is a Title I reading project and so references to schools, 
and children mean target area schools and disadvantaged children. 

c) A conpanion book to this training package is a collection of 

assessment instruments: Taxonomy of Evaluation Technique ^ for 

Reading Programs . svaiLable from the Measurement and Evaluation 
Center in Reading Education, 218 Pine Hall, Bloomington, Indiana. 




\ 
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Situation :-^ - ' - 

vou h.ve bean hired b,, Che Mr Unified School DleCrlcC to aeelet re 

tl.e development end evelnetion of a Title I reading project. (The hiatrlct 

hopee to net nOO.OOO from the federal government for the upcoming proJeoO 

' a i- T>r R Leader, vas especially interested in hiring you 

*Ehe superintendent, Br. R. Lena t 

becaoee yon poeseesed expertlee in both evalnation and reading. Hovever, 
he- informed you that your evaluation expertise vas the hey to your being 
hired, as the school district already employs an elementary curriculum con- 
sultant vho is considered an expert in reading. Specifically, you have been 
ashed to assist in the program development as veil as to guide the project 
through Its development. Your Job is not to develop this proposal by 
yourself but to utilise your technigues and shills of evaluation to monitor 

the program development and implementation. 

GIS is similar to many school districts of the same site and socle- 
economic Status. (See school bachground information sheet.) Hovever. one 
important deference exists at Cis. hr. K. leader is convinced that program 
development should he based on valid and reliable evaluation. He attended 
a national conference last year vhere Dr. Gubblebeam of Ohiena State Hniver- 
aity discussed the value and need for continuous program evaluation. Addi- 
tionally, Dr. leader has been getting increased demands for evaluation reports 
about his school programs from: (1) local groups interested in the school-. 

(2)- the locel Board of Bducation, (3) the State Department of Education: 

(4) the U. S; Office of Education. 

Vlhen you vere hired. Dr. leader told you '• that he vented to build a 
solid Title I reading program based on sound evaluation. Hovever. he also 
warneJ that some individuals in the achool and in the community would not 
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take too kindly to the idea of "being evaluated." Even though he vas 
supporting the idea of a broad concept of evaluation, he was advising . that 
you should not begin by gathering data "all over the place." He ended 
thiis initial meeting by saying, "You’ve got to show the staff that this 
evaluation stuff is not going to be directed at them. If they think you 
are criticizing their work, you’ll be in trouble, . and I won’t be able to 
ball you out." 

First day on the lob ; 

You have had thd following Itcans to read, some of which are attached: 

1. Title I project applicant book (not Included here) 

2. School background information (pp. 18-19) 

3. Job description (p. 20) 

While setting up office files, getting to know the staff (you’ve 



3. A letter to Dr. Leader from Ins Tructor, a teacher in. the district 
wiio took a graduate research course the past summer and used the 
school test results in her research project, (p. 23) 

A* A memo of a recent phone call Dr. Leader had from M^s. I. Rate. (p. 24) 
5. A memo from Dr. Leader concerning a previous in-service program. (p. 25) 



taken several coffee breaks), and studying the above items, you are 
interrupted by a call frbm Dr. Leader's secretary. She is sending you 
some items related to the reading program that Dr. Leader thinks will 
interest you. The following items arrive at your desk an hour later; 

1. An article froih the local paper (p. 21) ^ 

2, A' report from ^the local PTA (p. 22) 




O 
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APPLICATION FOR FEDERAL. ASSISTA^’CE 
for the Education of Children fr&m Low-Income Families 
under Title I of Public Law 89-10 



4 



L9* 









APPLICATION FOR FEQEm ASSISTANCE (continued) 
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JOB DESCRIPTION 

Title: District Evalua^tor 

Description: 

The evaluator shall be in charge of evaluation of all dlstrict^^itle I 
Reading projects and programs'. Because of the dual responsibilities oX 
his position, he must be an expert in reading content as well as in \ 

evaluation technique. The evaluator shall be involved in all projects from 
the formulatioji ■ of the project design to the evaluation of project end 
results. Moreover, the evaluator shall be in charge of preparing evaluation 
reports and disseminating the information to interested parties. 
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News Clipping 



JOHNNY CAN'T 
BtJT ELMER CAN 
IN BIG CITY 



GiK, September 13— Mrs. Mary Smity, 
a noted educator from Big City, 
was the guest speaker for the first 
session of the" Community Action 
Project’s orientation for the 
After School Tutoring Program. 

Mrs. Smith described a highly 
];$l^cessful reading program now ^in 
p^^gress in one of Big City's schools. 
'She very enthusiastic about the 

program. S-itjce the beginning of 
the program, there has been much 
more interest in reding and the 
students' reading sHA^ have 
improved. 

” Mrs. Smith described a^t^ical 
classroom. Children may choose~from-v^ 
a variety of books screened for 
reading level. The students are 
involved in charting their reading 



progress as they meet individually 
with the teacher several times a 
week to check their skills. 

"The variety of highly 
relevant materials, the individual 
work with -the teacher, and the 
continual reinforcement from 'the 
progress reports are factors which 
account for the success of the 
program," Mrs. Smith stated. 

The next orientation meeting 
for the After School Tuforing 
Program will be held Oct^ 22 in the 
community building. 
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Report to Superintendent Glz Unified School District 

FROM: Parent Teachers' Association, Kennedy School 

SUBJECT: Annual Report from Visitation Day 



Dear Dr. Leader: 

During our visits last week to other school districts, our committee 
was much impressed by the reading program at Neighboring Unified School 
District. They have an excellent remedial program and a very wei 
planned and interesting in-service program. Many of the teachers are 
experimenting with iudividualizod Furthermore, the students 

seemed interested in reading, and many ware checking out books to read 

at home. 

Naturally, our committee noted c’.r i?chool district also has an 

excellent program, but it might be worthwhile to consider some of these 
innovations in the reading program. 



Respectfully yours, 

/i/ 

N. Volved, President, PTA 



Mrs . Smith 
Mr . Jones 
Mrs. Rogers 
Mr . Brown 
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14 University Way 
Marblehead, Indiana 
August 12, 1969 
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Dr. R. Leader, Superintendent 
U. S. D. of Giz, Mo. 

161 School Lane 
Giz, Mo. 

Dear Dr. Leader: 

Thank you so much for allowing mo to use the District «st scores In 
mv research project this summer. The scores were most helpful and^lent 
themselves well to the B582 research course. You may recall ‘ , 

p^^ct on which i was working was a longitudinal comparison of aptitude 
Lst results with particular emphasis on “f’ 

apparent. I thought perhaps ywu might be imterested xn the results.. 

s;r: "S ™ 

norm witn a sixgnu - cmrea in tte" socxal 

11 1 t^hpT•e liflSj Si (JefiLnExt® drop xti scores 

sLdiea’, reading, hisi:ory, antf BngLish Snclu- 

vocabullry, tnath, a^d science Te^Tln coun ts 

math, and science sections thain In the other areas. This factor may o y 
not be the cause of the drop in the other scores. 

I'm looking forward to the start of school and to being a teacher 
again instead .of a: student! 

Sincerely, 

(Mrs.) Ins Tructor 



ERIC 





2B 



I 



24 



, . from the desk of 



R. Leader 
Date August 18 



To: 



Val U. Ator 



Subject: Mrs. I. Rate 



Another call from Mrs. R.— sieems she read Flesch 
this summer and is all upset becatuss Todd (2nd) --isn't 
reading the newspapers yet. Wants phonies t !3fou'll be 
hearing from her ! Good Luck . She carries a lot of 
weight in Benton School as president of their P.T.A. 




O 
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GIZ SCHOOL SYSTEM 
Interdeswartmental Co tnniu nica t ion 



TO: val U. Ator 

SUBJECT: In-service program 

FRCW: Dr. R- Leader, Superintendent 

I understand through the grapevine that there was an Ln-sarvice 
program about 18 mont3as ago, before I^caane to the School System. 

All past records concerning this project s^aem to hav^a been mislatdr 
My secretary has izif opened me that the focxis was on corrective reading 
for the disadvantage. You might check this out as possible aid 
to our upcoming project. 

f 

Sincerely, 

. 

Dr. R. Leader, Supt. 





2S 
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Second day on the job : 

You’re called to Dr. Leader’s of£ii=e to discuss the Title I project. 

-DEr. Leader laaSces it quite clear thet th® proposed pro^ecc Is to emphasize 
iHdividnallzed reading. He does not seem to be quite ^ra what individualized 
reading is, but says he want yom to find out what the «c*ool district is 
doing and should be doing in this area. In his convetsaatlon, ha refers 
«.everal tltaes to the article he sent y. u, -’Johnny Can* ' but Elmer Can in 
Big City," and also to the PTA report. He tells you to ^consider a plan for 

getting started. 

Third dav on the job ; 

Dr. Leader phones to discuss what you're doing t® get started. He 
states that he knows how important evaluation is to good program deve op , 
and he wants a good evaluation to deternnlne the district's progress in 
individualizing reading instruction. Htowever, he emphasizes that time is 
going by and that the proposal must be ready by January 15 . He asks if you 
can send him a report in four days outlining what steps you will take to 
start the program rolling. He seems somewhat irritated and ends the 
Conversation by stating: "I hope you know we're on the spot in this thing. 

I can just hear them all hollering next spring if I don't have these 

evaluation reports in on time." 

A oema arrives from Dr. leader the next momlns; 



2iJ ■ 
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GIZ SCHOOL SYSTEM 

Interdepartmental ConEmmication 

TO: Val U. Ator 

mOM: Dr. leader 

SUBJECT: Title I Project 

Sony if I sounded upset on the phone, but I had just received 
a copy of the Title I evaluation form from Washington. I can't>-^ 
understand It at all. In any event, we'll probably have to complete 
something like this for the project proposal; but I don't think it 
follows the Idea we've been talking about. 

I-That I need from you is a report which includes— 

1. A list of steps you're planning to take to detertnlne needs 
in the area of reading we discussed. 

2. Some procedure we can follow to assign priorities to the 
needs. The district has gotten into the situation in thtf 
past of trying to do all things at once, and I don t want | 
that to happen again. 

. Do you think you could have this ready by next Wednesday? ^y 
the vay, I thought you night like to visit the elenentnry schools 
to get to know the teachers and that sort of thing. I really don't 
think they'll be much help to you because they don't know much 
about evaluation. They still belive in standardized tests as the 
ultimate answer, and I think I know where you stand on that. In 
any event, the principals will be looking for you on Thursday and . 

Friday this week. 1 think you'll enjoy meeting some of our 
principals. You'll like Mr. Crabtree over at Benton School; he 
has the Secretary of the Year, and he's a wonderful morale builder 
in that school. Anyway, you might like to browse around and have 
the principals show you the facilities. We. have some really fine 
facilities for special services. We're very proud of them. 



30 



RESPONSE SHEET 

In view of Che two ehlnge that Dr* leader aaUed you to have ready 
by next Wednesday, what notes would you atake to yourself? These might 
be very brief plene or eoine general errategy to fulfill Pr^ Leader s 

reque3t« 











\AT WHAT POIHX IM SHE PIPP PR0G8AM 
YOU NOW? . 
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OJIHISCIENT COHMBHT 



In order to deCemlne needs* you mlgbt heve -■ 

the following areas: 

1. what waterlels are now being used to 'teach reading? 

2. How do the studehta perform on standsrdired tests? 

3. What do the auj/ervlsors and teachers think the system needs? 

4. Is there fioum conmunity group that speaks for the schools 

involved with the Title I project? 

(YOU may not be able to do all that by next^ Wednesday, but you at 

least make these notes on your pad.) 

R, IMder aUo w»nts you to set up procedures end esslgu priorities. 
Thsi one Is . stunper. vn,ere will yod tlnd crlterU to esslgn priorities? 
You might have listed: , 

1. Does research say where to” start with disadvantaged populations? 

2. Should I ask for a coamittee to ‘set criteria for priorities? 

. They would know the school system better than 1. 

- 3. Has the reading supervisor sent out a questionnaire that would 

help? Better see him. . ^ 

(That’s right. Your alternatives seem quite tenuous at this point. 

You need more feel for what is going on.) 

~ individualizing reading Instruction does not revolve around how 
one teacher tutors one child. As a matter of fact, the teacher cannot 

consider himself a private tutor. ' ‘ 

Individualizing will hecessarily involve the proper identification of 

Student needs and the organization of students, teachers, wai^als. 



facllltiea; the schedule, end perhaps the aelsbborheod connamity. At any 
rate, the evaluator most eacamlne each of these elements as they presently 
exist to see if they are capable of promoting a greater Indlviduallaatlcn 
of instruction. In effect, he asks. "How is the classroom organized?" 
"Boo'can I coliect that information?" 












- AT WHAT ponn: IN TBDE CIPP PROGRAM 

ARE YOU HOW? Contexts Evaluation; Focusing 



Thursday and Friday 

You visit the schools and engage in conversation with several teachers. 
Over a cup of coffee at Benton School you listened to one teacher who 
registered a description of rather t3rpical concerns of other teachers: 

"Mr. Ator, you’re just the person I need to talk to. All semester 
I’ve been tearing my hair out over— a child in my room -- Fairlee is his name 
who is so disruptive, especially during reading, that I'm making no headway 
with my other students. And.^ny of are really fine children who want 

to learn. You can’t imagine what-<^^s like to have Fairlee in your room 

/ 

for a full six hours. He's constantly tapping his foot, or his pencil, or 
anything else he can lay his hands on: paper, or paper clips, anything. 

You get the idea. And when it is his turn to read, he obviously isn't 

"V- ' ' “ ' 

with us. I have to take time out to help him find the place, sometimes 
even the right page. Well, it takes so long to get Fairlee settled down 
and with it, that by the time he's ready to begin reading we all-know he 
isn't going to be able to read the material anyway. I think he ccml'd, if 
only he wanted to. He’s not really an unintelligent child. Oh, another 
thing, he doodles. This at least is quiet, so I can live with it. But, 
in addition, when I’ve assigned paper work, Fairlee is out of hia chair, 
every other minute, needing to talk to me, you know: *What am I supposed 

to do with this? I don't understand what yoii mean. * Xte you have any 
suggestions as to how I might handle this child? I really am at tay 
end." ■ . 

Teacher B ; "r'm interested' in what you have to say about this Mr. Ator 
since I also have a problem child. His difficulty is quite different, but 
I might be able to use the same technique," 
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In other schools, the feces were different but the complaints were 
similar. Almost universally they focused on some child who was a behavior 
problem during the reading period, or one who showed complete lack of 
interest in the materials that were used, or another who just dxdn't have 
the vocabulary to keep pace in the readers provided for tlie class. The 
teachers didn't discuss general group performance; their eyes were set on 

ah example of a need -- a child with a problem and how they could help. 

! . • / 

I You have gained knowledge and experience from your visits to the 

! • f 

schools. In addition, you know Dri Leader wants to emphasize individualized 
reading. What steps will you take to determine needs? What procedure will 

you suggest to assign priorities? 

'* ' ■ ’ ■ \ ' 

I' . . /■ 



I 

I 
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RESPONSE SHEET 



1, Steps to take. 



\ 



2. Procedure to assign priorities . 



O 

ERIC 



AT WHAT POINT IN THE CIPP PROGBAM 
ARE YOU NOW? 
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Suppose fchst: your answer to R« ZiCader was the following iuenio: 



’SO: rI Leader j Superin teadent SATE: Wednesday #2 

FROM: "'-1 U. Ator 

SUBJECT: Title I Project Planning 

In response to 3 rour request for an initial plan of action, I 
propose the following answers: 

1, Steps to determine needs in individualizing instruction. 

1. Visit schools for informal reactions of teachers and 
principals. (Done last week). 

2. With help of reading supervisor examine existing 

reading texts to determine provisions for individualization. 

3. Do a spot survey of other reading materials in the class- 
room to see how they can be used for individualizing 
reading. 

4. Find out from conanunlty action group what kinds of personal 
instruction they see their children needing and what kinds 
they see them actually getting. 

5. Do a small sac^le check on the teachers' background for 
iadiv* dualized reading instruction. 

6. Check to see if there are any tests that would Indicate 
areas to esiphasise 1^- trying to individualize instruction. 

/ 

2, Procedure to assign priorities. 

1. Establish a committee composed of the superintendent, the 
reeding supervisor, the evaluator, four teachers, and two 
representatives of the community to establish the final 
criteria for selecting the priorities. 

2. Hold five or six discussions among teachers in various 
buildings to see how they would list the priorities for 
individualizing reading • instruction -- facilities, 
in-service training, and othev services. 

3. Have a sasiple group of teachers rank the needs of their 
Title I students -- their needs related to more individual 
attention. 



AT WHAT POINT IN THE CIPP PROGRAM 
ARE YOU NOW? Context 
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The next day D?. I^eader responds to your memo with a telephone call. 

"Val, your plan looks fine. Let’s get started right away. I think 
we can give you about one month to get the assessment job finished, and 
I’ll see that you get some help in collecting the information that we need- 
Miss Overth Hill is the reading supervisor. As a task force, she and two 
of her reading teachers should be able to help you collect all the Information 
you need within a month. In the meantime, 1*11 get the committee set up to 
help us decide priorities. We’ll call it the Title I Curriculum Committee. 

You and your task force will be on the committee; I'll get the others together 
shortly. Oh, by the way, Val, any meetings that you hold with classroom 
teachers must be Informal — no public announcement. The union is already 
demanding decision-making power over salaries and the like. If we hold 
formal discussions, we’ll have to do it every time we want to try some new 
project. Maybe you can tap the teachers through some kind of questionnaire 

or something. " 



O 
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RESPONSE SHEET 



It is now youir job to find in one month what the specific problems 
(needs) are for individualizing reading instruction. Using the steps 
that you outlined in your memo to R. Leader « what specific things are 
you and your task force going to look for? What instruments would you 
use? (Check Taxonomy of Evaluation Techniques) 



Specific Information 



Ins trumehtat ion 






AT WHAT POIKE IN THE CIPP PROGRAM 

ARE YOU NOW? 

„ 39 
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OMNISCIENT COMiENT 

Students. Are the students all performing to the best of their 
ability? How many are failing reading or other subjects because of 
inadequate reading skills? Of thos^ who fail, is there a significant 
discrepancy between their reading and their ability? Check school test 
records and interview;^ teachers* 

Teachers . How do teachers define individualized reading? Do they 
see themselves directing it? With how many students? The teachers had 
an in-service -program eighteen months ago. Did it provide any skills that 

would help tndlvlduaUze reading Instruction? How fisany teachers In the 

S 

Title I schoole participated in that course? 

Perhaps, through inteirviews you could determine whether the teachers 
have any knowledge of how to wjte information from group reading tests, or 
how many could use an laformal reading inventory. Probably through a 
questionnaire you could ask the teachers how they view facilities, materials 
school organization, and their own competency for Individualizing reading 
instruction. See examples of instrumentation in the Taxonomy of Evaluation 

Techniques . 

“ Materials . To individualize reading. It would be necessary for tne 
teacher to have available a wide variety of materials, written at several 
levels of difficulty — appropriate for the cultural groups that make up 
the population. Some hardware such as tape recorders and film-strip 
projectors would also be beneficial. A spot survey by a member of your 
task force should be able to estimate the availability of these items in 
the target schools. There are materials checklists in the taxonomy. 

Paeilitles . The main concern with facilities is that they be flexible 
and, of course, available. Can chArs and book cases be shifted around to 
provide for various seating arrangertwnts? Are there places open for 

40 
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different klnd^ of activities? Is there a library or a learning center? 

Are' there carrels for private work? Can tenqjorary partitions set off 
comers of the room for specific activities? 

Existing Program . Does the existing curriculmn include individualizing 
reading as one of its objectives? Does it suggest specific plans of actlro? 

If so, why hasn^t the plan worked so far? 

Coi^nMOtty . The community xnay i>rOvide activities or encourag^ent for 
individualizing reading using lending libraries, neighborhood tutors in 
reading and book clubs to get the right book into the hands of the right 
youngster. If they do not exist, would the community groups be willing to 
develop some of these activities and services? 

The point . In the time you have for working on these exercises in this 
simulation package, you could not be expected to list all the things that were 
mentioned on the previous pages. It should be clear, too, that there are many 
other questions or suggested sources of Information that could have been listed. 
The point is that a fairly complete description of the existing facilities and 
of the evident needs should be produced in order to decide on priorities for 
program planning. 

You remeaiber that, at Val’s suggestion. Dr.- Leader set up a Title I 
Curriculum Committee to establish the final criteria for assigning priorities, 
to the determined needs. In addition. Dr. Leader organized a task force 
composed of the reading siq>ervlsor, two reading teachers, and Val as chairman 
to determine what the needs actually .were.- These two committees went to work 
immediately. We return now to Val and his task force to see what needs they 
found in the Olz system. 




at w?at pomr us the cipp program 

ARE 'iCU NOW? Context; analyzing data 
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TO: 

FROM: 

SUBJECT; 

1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 



GIZ SCHOOL SYSTEM 
Interdepartmental Communication 



Title I Curxioulum Committee 

Task Force on Reading Needs, V. U. Ator, Chai rman 
Needs for Individualizing Reading Instruction in Giz 



A random selection of teachers from grades K-6 was found to 
have very positive attitudes concerning the need to individu- 
alize reading instruction. 

A questionnaire survey of this random sample of teachers 
revealed that over 90 percent of the teachers fesl that their 
graduate and undergraduate teacher training has not given 
them any training in the individualization of reading instruction- 
This feeling of inadequacy covers reading materials, instruc- 
tional practices, and grouping procedures. 

In most of the classes (over 80 percent) the children are 
divided into three groups for reading instruction. All three 
groups within a class use the same basal reader, but modifi- 
cations are made for each group, primarily in the supplementary 
material used, the speed with which the children go through 
the material, and the depth of discussions regarding the 
comprehension of stories. 

There are a number of sets of high- interest, low- vocabulary 
books available in each elementary school library. However, 
these are not being used by classroom teachers. Many teachers 
arc not even aware of their existence, 

A landom testing of ten children from each grade level in each 
school revealed that over 50 percent of the children were being 
given reading instruction 'in materials which were at or over 
their frustration reading level according to the Bett’-s criteria-. 
Ten percent of the children in this same sample were reading 
instructional materials at their independent reading level. 

iifcudy of last year's standardized test scores in reading 
revealed that the children in grades 1-6 in the Giz School 
District had an average score at about_the AOthf percentile 
for each'^grade. These results corroborated the findings of 
the study done last summer by Mrs. Ins True tor, 

A study of PTA committee reports. Board of Education minutes, 
local newspaper articles, letters to the editor column, seven 
visits to the local coffee house, and two haircuts at the 
community barbershop reveal a generally positive attitude toward 
the school program in general. However, most people are quite 
uninformed about the school program in general, as well as 
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RESPONSE SHEET \ , 

^ , * 

r. ‘ • 

The task force's report listed eight items. Describe .(create) two 

other pieces of relevant data that the task force might liave collected 

If they had covered other areas of possible needs for individualizing , 

> ■ , 
reading. (Think about needs in your own' school) 

9 * 
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The Title I Curriculum COnniit tee accepted the "Needs T«k .Force Report^ 
and produced the following report of its own- 



Report from the Title I 
Curriculum Committee 

The following goals and criteria should govern the Title I Reading 
Program for the G±z Unified School District. 

Broad Goal 

■ Students should demonstrate continuous basic reading skill development. 

They should also be exhibiting positive attitudes toward reading. 

In order to reach this broad goal, the curriculum committee proposes 

the followieg specific objectives: 

/ 

Every student should be given reading instruction at his Instructional 

reading level- as determined by an informal reading inventory. 

A report to' this committee by our Title I Reading Evaluator reveals 
several discouraging facts about present attempts to reach this goal. We 
are, therefore, appoiiiting a task force of this committee to propose 
alternative strategies for closing the gap between our objective and our 
actual situation. The task force should clearly delineate the strength^ 
and weaknesses of each alternative according to the criteria below. Val U. 
A tor has been "asked to serve as chairman of this group. Their report is to 
be presented to this total, committee by November 15. 

Er|c r 
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Criteria, for Program Selection 

1. The pr og r a m should deal directly with the problem o£ get ring 
children in appropriate reading levels. 

2. The program should have lapacC in all elementary grades for 
all schools, 

3. The program should make individual Ira tion of reading an effective 
force in everyday instruction. 
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RESPONSE SHEET 



In light of the asseismsnt and diecussions that have taken place, are 



there^ criteria that ahoul|d\be added to the Curriculiun Coflnni.tfcfce'e report? 



4 « 
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\ 



\ 



5 . 



6 . 



Are ^ there any changes? you can suggest fpr the first three criteria? 



AT WHAT POINT IH THE CIPP PROGIIAM 
are you NCK-7? ■ - .-I ■ ' : 
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On JToveinber 1^ the Title I Curriculim Coiranittee net to review the following 
report on altemativie program suggestions^ 

Task Force Report 

Alternative Tor Tilndividualising Reading in Giz 
(Veil U, Ator, Chairman) 

On the basis of the priorities established by the Title I Curriculum 
Committee, the special task force has produced five proposals for 
individualizing reading instruction in the School System of Giz. They 
are presented here with their major strengths and weaknesses • Thas process 
should assist the committee in making a decision about the kind of program 
that it would choose for Title I* The task force made use cf research 
studies and relied upon the broad experience of Miss Overth Hill in 
listing these strengths and weaknesses* 

The task force also made some informal decisions about the relative 
merit of each of these alternatives. It felt that the alternatives 
could be ranked according to their merits for Giz in this order: 

1 • In-service program for all Title I reading teache.rs on how to 
individualize reading instruction. 

2. Provide $2^000 for each classi'oom teacher to purchase the materials 
and the assistance that will benefit him most in individualizing 
instruction. 

3* Hire a reading speciali:3t to work with those children who are 
considerable handicapped in reading. 

U* Hire paraprofessionals to assist each teacher in the classroom. \ 
5.. Hire a specialist to work with teachers as iin on-the-job trainer 
in individualizing instruction in reading. 

6. Conduct a special summer school for those who have trouble with 
reading. 

7* Reduce class size from 30 to 25« 



Each of these suggestions is given below with an outline of their 
strengths and weataiesses. The task force will be happy to explain any 
of these alternatives* 

i 

Signed. 

Val U. Ator 
Overth Hill 
June Sraartack 
Bjorn Looser 

Dated* Wednesdaya November 1 5 
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S'breng'tJis and Weaknesses of AX'fcema'bives 
to liidivldualizing Reading Instruction 



Alternative 1 • Specialist working wi th children 
Strengths 

1 • Expertise is applied to those who need it. 

2. Small groups permit highly individualized program. 

3 , It frees teacher to work with average or better 
children. 

U. Tangible, immediate gains are likely. 

5 . No additional demand is made on teachers* time. 

6 . The low cost is limited to the specialist’s salary 
plus few materials. 

Weaknesses 

1 . Only a small number is ^ ' ^ted . 

2. Space is needed for group. 

3 . Scheduling would present problems. 

II. Finding qualified personnel is difficTilt. 

$. It leaves teacher out — no change in institution. 

6 . Children may feel isolated as poor readers. 

7 . Teacher may feel using specialist labels him 
as poor t03'C3'ier. 



Alternative 2. Specialist iworking with teachers 
Strengths 

1 . It involves large number of teachers. 

2. It involves large number of children. 

3 . It places responsibility on teacher. 

U. long term effect on institution is likely. 

There is no space as well as presenting scheduling 

problems . ... 

6 . Labeling and isolating poor readers is avoxded. 

7 . There is a low cost (specialist's salary plui few 
iE3.terials ) » 



Weaknesses 



1 . 

2 . 

3. 

5. 



Expertise is spread tliin. 

No immediate gains are likely. 

It makes demand in teachers’ time outside _ classroom. 
Finding qualified personnel would be difficult. 
Specialist’s personality must be effective in 
working with teachers. 

Difficulty of coping with full range of reading 
abiXity from no ability to evperior, in one classroom 
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Al-bernative 3 



Sturaasr School 
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Strengths 

1 . It gives intensive help for poor readers (12 students 
per class). 

2. The space anH scheduling problems would not exist. 

3. Teachers could experiment vrith new ideas. 

I 4 ,. Students may improve more when removed from the 
regular curricul\im» 

V7o sldio S S 6 s 

n iiath 12 pupils per class, the program could reach 
only a few of the poor readers. 

2. Attendance may be a problem in the summer. 

3 . Students may regress if the regular program is not 

; changed to continue work started in summer. 

Alte mati ve 1;. Reduce Glass Sise, 30-25 

Strengths , _ 

1 . The teacher would have more time to spend with 
each child and a better opportianity to follow 
the stiodents* progress. 

2. More space and seating flexibility coxxld be used. 

Weaknesses 

1 . Instruction may not improve. The teachers usually 

do not individualize on the basis of a ? pupil 
reduction. • 

2. More classroom space must be made available. 



Alternative 5. Hire paraprofessionals 

Alternative 6 . M aterials s $2000 per teacher to be spent by teacher 
Alternative ?• Iii"Service Program 
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RESPDl^ SHEET 



If you been part of the task force to find alternatives for 
individualizing reading, what are ^ the ma^or strengths and weaknesses 

T 

that you would have ^listed for alternatives 6,- and 7? 

Altenaative 

Strengths 

Weaknesses 



Alternative 6 » ^ 
Strengths , 



Weaknesses 



Alternative 7 « 
Strengths 



Weaknesses 



At what point in the CIPP Program 
•are’ you now? 
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OMNISCIENT COMMENT 

IdBted belmr are some of the strengths and westossses that you sd^t 
have given for the alternatives suggested by the task force headed by the 
evaluator. On your oHn list you nay have indicated sone that are nore 

appropo than some of these . . 

ftt-hi^rnative 5. Hire paraprof essionaj^ 

teacher would be freed to spend more time 
’ Sd w^k5.n6 individuals^ ^th the students. 

2. The paraprofessional could provide more-one- o- 

3e ^lationship ^ri.th the 

provides more adult contact for the student. 

' ^ - ^^S chers do not knoi^ how to use assist^ts. 

2*. ^ere be a problem finding ,and training 
qualified people. 

Alt.amative 6. Materials «. $2000 per^ ^^r to be spent by teac t^ 

§J^2BS— Qj^Qourages experimentations and -the new ideas 

2 It^in^l^s^tS^in^vidual teachers, and 

they w^d be more enthusiastic about changes in 

3. S^ets the specific needs of each class and 
teacher more completely. 

"^^ ^^"jl ^jould be difficult to evaluate the progr^. 
zl Teachers may need new ideas rather than tiwre . 

3 . it^Sy^e* difficult to provide for transition 

li,e Te^hers^^”not prepared to select materials . 
oT personnel* 



Aitornative 7. In-service Pro^^ 



gets new 1. fTormeticn to the pecple who are in 
a position tc apply it. \ 
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At what point iii the CXPP Progr^ 
are you now? Input t alterna tlj^ 
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2. Fulfills the needs of teachers. 

3« "Inspires." 

ii» Acts as a motivational factor. 

Provides new ideas. 

6. Helps -fiioae teachers who would not otherwise 
get any further schooling. 

Wealmesses _ . 

n nfTT nw will not approve after-school hours. 

2. Instructors who relate program to the needs 
of teachers are iiard to find. - 
3» Hiere is no good, central location for nieeting. 
k. Immediate effectiveness of program would be hard 
to meastire 

Teachers ai\ metimes reluctant to devote extra 
time. 
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Cuiri cult3iii Coninii'fci^© / ITi'fcX© I Reading 
Meetdngi November l5 



On November l5 the Ciirriculuni Committee reviewed the alternatives 
and px-iorities suggested by your task force. The comraittee tharil^ed 
the task force and then discussed -which alternatives should be 
selected for a program. At first the ' coirafeLt-bee decided that it woiild 
start "at -the -bop" and devote its energies to teacher training and 
to providing assistants -who would help build the skills and attitudes 
of individualizing reading instiniction. Bat the comnninity actien 
representatives felt that each child in the target schools should 
receive soms direct benefit from the $100^000. For that reason, the 
coCTitdttee decided to include the alternative of each teacher in the 
target area purchasing extra materials . Thus the committee concluded 
by saying that the Title I Reading Program for Individualizing 
Instruction should incluc's.^ in order of impcrvances 1) in- service 
training for target school teachers, 2) the hiring of resource teachers 
to aid in carrying out the developments of the in-service training, and 
3) -the purchase of extra student materials by the hers in the 

classrooms. 

The Curricul\im Committee then submitted the following objectives 

for incorporation in the prograin: 

1 . To ir^rove the reading skills of every child in the target 
area in grades 1, 2, and 3- Special emphasis was to be 
placed on early prevention problems. 

2. To produce -teachers who use diagnostic -techniques for teaching. 
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3. To make teachers aware of cirrreat materials available for 
individualising reading inst*ruciion^ c 

nxe committee hammered out its objectives in detail and submitted 
them to you for your reaction. 

Objectives are on attached jsbeat. 

r 
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Objectives for Title I Erograra 
Individualizing Raaxii^ Instruction 



Objectives related to -the in-service programs 

1. To increase the teachers* knowledge of basal readers> kits, 

A-V tools, and tests (individual inventory, group diagnostic 
tests, and achievement and intelligence tests). 

2. To increase the teachers’ knowledge of the method of diagiiosoic 
teaching. 

3. To institute the teachers ' use of informal invent-ories. 

U. To further the' teachers’ use of individual student files of 
reading progress. 

3 . To promote the application of the teachers' knowledge of 
sequent!-" skill development. 

6. To institute the , teachers ’ use of varied types of reading 
materikts and instructional tools. 

7. To irici'ease the reading level of each“child to a ‘greater; degree 
than Changes noted in past years. 

Objectives related to materia;- -ss 

1 . To pro'vide materials which are written at the appropxdato 

„ reading levels for each individual child in/the classroom. 

2. To provide materials which may be used for the development 
of a sys'tematic sequence of skiHls. 

3. To provide mateidals which vary in content, type, length, 

and. interest. — 

U. ' To increkse|tlw number outside books read by the students. 

Objectives related to resource teacJierss 

1 , To hold montlily' indivi^^ conferences mth each classroom 
teacher to discuss Individual problems and offer new ideas. 

To observe each "teacher period cally to assess and aid in the 
individualizing, of reading instruction. 

To acquire needed classroom materials wliich -i^ill enable each 
student to read at his instructional, level. '• 
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RESPONSE SHEET 

What do you think Val*s reaction would be to the sijecific objectives 
selected by the Curriculum Caramittee? What is neoessaiy for him. to 
develop a good program plan? 








0 



At polut in the CIPP Program 
axe you noH? ' V 







OMNISCIENT COMMENT 



Val told the Crjiriculum’ Oammittee that a program could not be 
written on the basis of the objectives provided by the committee. "We 
wouldn't know where we were going I” he told the ccannittee . "Let me 
draw up a list of more specific objectives— objectives ws can live 
with arift develop a program fnom. If those objectives meet with the 
approval of this coraDiittee, then we can get the program described 
and moving." 

Val was given the job of developing more precise objectives. 

He was to have written a report to the committee by the following welk. 

***■«•****•*<•*** ****** 

Val's Report 

Specific Objectives for Title I Reaxiing Program 
It was decided that the proposed Title I program tTOUld have three 
coTi5>oitehts: a formal irservice educatioi^ resource teachers 

for continuing assistance, and materials for direct use by the youngsters 
in the classroom. Bie major goal of the pro^am ^ to teach teachers 
about the diagnostic teaching of reading and. th^'use of instructional 
materials that will assist in individualizing rea^g.: The following 
specific objectives are an attentat to incorporate both the general 
objectives and the intent of- the program in. a 'list that can be used to 
develop procedures fpr carrying out the program. 



At Tdiat point in "ttie CIPP Program 

are you now?,, Inputs Specifying objectives 
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Ob.iectiVBS Jor the ELtle I 
Reading Krogran 

In-service edxicabion for teachers 

As a result of various in-service activities for teachers in 
grades one to three it is expected that the teachers will be able to— 

-o‘ ' ‘ 

1 • Ids’t and deseidbe the specific needs of disadvantaged children 

i 

in their classes. Needs of children in the following areas 
will be included? ^heral interest, social, academic, and 
emotional. The definition of disadvantage^nt as given in 
the Title I applicant's book will be used for this study. 

2. List define a developmental list of reading skills. 

3. Define specific criteria for determining students* functional 
reading levels. 

U, X Select, develop, and adniihister a variety of measurement pro- 
^ cedures and Instruments for de'^rmlning each child's instructional 

level .and skill development as defined in obje’ctlves 2 and 3« 

5. Select and demonstrate the use of a variety of rea^ng materials 
that could be used' to develop each j^ld's reading level and 

skill develojmient as . defined in objectives 2 and 3« . 

' / £ 

6. Select and demonstrate the use of a variety of "reading materials 

. ■ V . X. 

that are appropriate to the needs (interests, social, academic, 

> • and, emotional) as described in objective 1 . ‘ 

7. Demonstrate the use of a variety of instructional procedures 

■ ’ ; that are appropriate to "ttie needs and ;reading skill (teveloprasnt 
of each child. 
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8« Design a teaching strategy IJiat will provide the insti*uctor vlth 
inraediate feedback of each student’s performance of the reading 
sl^Us listed in objective 2. 

In addition to the in-service education of teachers , the project will 
provide funds for purchasing \^^tructional materials which can be used 
to individualize reading instruction* The objective of this phase of 
the program is to provide the ’’appropriate" materials for each classroom. 
Specifically, the objective is to make avaoilable to each classroom a 
variety of instructional materials which— 

1* Are appropriate to the specific needs of 'disadvantaged children 
in that classroom* 

2. Include x>ractice activities for the, specific reading skill needs 
of the children in that dassroom* 

3 • Are of the appropriate functional reading levels for the 
childx*en in that ciiSgsroom* 
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Tour report; is sent to the Curriculum Ccsnraittee on schedule. Due to 
a stomach disorder you are unable to attend the meetingj but, you find 
this memo iraiting on your desk when you return to the office s 



aiZ SCHOOL SYSTEM 
Interdepartmental Communication 



TO! Val U. Ator 

FROM! Dr. Leader, Superintendent 

SUBJECT! Reading Program, Title I Project 

Sorry you missed the Curriculum Committee meeting last 
night. Did you have those^ ulcers "when I hii^d you in ‘ 

September? , ' 

Your detailed objectives have been accepted by the Curriculum 
Committee. A program task force con5>osed of the elementary reading 
teachers and Dr. Readbetter, a specialist from Ohioana, University, 
■will use the objectives ■bo develop ■the * t*ogram. As soon as you 
get' a .copy of the program, let me know at you ■think. 

\ - i 

I am pleased -wi^bh the ■way ■things ^ progressing. 



/^.S, V 




R. L - ''O'. X,- 



.1 



Two 'weeks la'ber you receive a copy of the proposed program. You must 

o' ^ 

read it and react ■t^ Dr. Leader. Time is getting short. The proposal 
deadline is only.-a few ■weeks away. ; ’ 

^ ■ -4 ' ■ 

■ . ■ . " .. • ; ■ 
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(Task Force Eeport) 



mCXSiMl ELAH 

Title I Heading Project on 
Individualizing Reading Instruction 

T 5 xe intent and the specific objectives of this program are 
found in. the report by Val Ator» Biis plan provides the guidelines 
to make the program optional. 

INTRODUCTION. The intent of this program is to upgrade the knowledge 
and skills of the primary grade teachers in four elenentary schools 
•in the t^get area. The purpose of the prograir^ is individualizing 
reading instruction. ,‘Ihough the teachers receive guidanos in two of 
the three con^jonents of the plan^ the ultimate goal of that focus is 
to enable an stxadents in the target area to read at a higher level 

O 

than they are presently doing. To keep the student in mind, a third 
component of the ptogcasai is a materials purchase plan whereby each 
teacher in the project has a sum of money to invest in classroom 
books and materials that will make reading more real and desirable 
to the disadvantaged children that he teaches. 

Each of the %hree components will be described separately even 

a C 

though they are closely related’. The parts form a sequence which makes 
it fairly siir5>le to discuss -ttiem in separate units. The in-service 
training cQH5)onent will provide a h^^ knowledge a n d an opportunity 
to develop some essential skills for individualizing reading. Following 
the in-service training, it is the respoissibility of the resource teachers 
to aissist the clas^oom teachers in carrying out and exteirfing the 
concepts and the skLlJn developed duping the in-service meetings. . With 
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tihe giiidelines provided ly the in-service 'training and "the recommenda-^ 
tions of "the resource teacher, the classroom "teacher ■STill "then have' 

V- 

j 

adequate direction for choosing ma-terials "wisely. 
in-service Component 

iVenly-ei^t primary grade "teachers from four target schools 
will^ pgrticipa"te in an in'tensi've and esrtensive program on indi"vidu|^- 
izing reading instruction. 

Personnei o Under "the direction of "the Giz reading supei^sor, 

Overth Hill, "two consultants frbm ,ou"tside "tdio school dis'trict -will 
plan and execute "the specific "basks of "the inservice program., One 
consultant shoxjld:be a reading specialist, jurohably from the Ohiana 
Unl"VBrsity Reading Depar'tment* second consultant should have ^ ~ 

badkground in fading ^ejiucation and -shoiild have a specialty in cur- 
ricular programs for "the di3adv^"baged or ±nner-ci"fy children. 
Occasionally speakers may be brought in to "treat specialized con- 
"tent or to : provide motivation. , 

lijne. The program will be conducted in nine full d^rs Of acti^"ty 
di"vided as follows o"ver "the academic yeari one full day a week for "the 
first five- weeks 5 one half day a mon"bh for "the remaining eight months. 

Place . , Initial meetings will be held at the Board of Education 
central office conference room. At the discre"tion of the consultants and 
accoxsling to the needs- of "the "teachers, subsequent meetings may be held 
in the "target schools or 6"fcher appropida-fca places. 

Content. The Giz reading super"visor and the "two consultants 
"Will prepai^e a de"tailed con"fcent outline at "the start of "the in-service 
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program. Generally -ttie program will be geared to acMeving the cozir 
cepts and skills listed in the objectives for the in-service program. 

The content vill include, therefore, a treatment of tests, materials, 
teaching plans and techniques fbx’ individualizing reading instruction 

for disadvantaged children. ^ 

Procedures . The in-sexvlce program will consist of a variety 
of learning eaperiences, each designed to achieve knowledge ov a 
sld-U related to individualizing raaiding. To make the program effective, 
consultants will use these procedures as guided by the contents of 
the programs 

1. Lectures on substantive issues, such as researph evidence 
on characteristics of disadvantaged learners. 

2. Demonstration of teaching and testing techniques, using 

children in the demonstrations. , r 

3. Small group/largs group discussions, depending on the 
mcessity for total involvement and reaction by participants. 
Workshop activities to consiaruct evaluation instruments and 
teaching tools. 

A book and materials fair to displ^ items useful in indi- 
\ • ' ■' ' ' . ■ ' ' ■ 

'y vidualizing instruction to give teachers an opportunity to 

practice with some of the instruments. 

6. Supervised testing session with one child in which the ..participant 

administers an informal inventory of his own constructim. 

- -X 
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Resource Teacher GcnnpouBnt 

6 » 

Four roaster teachers ■ 
schools in the project, for /(he purpose of assisting the classroom 



Fom* master teachers wili be hired, one for each of the elemantaiy 



O 

ERIC 



Go 



teachers on a contdiming basis* These master teachers will be hired 
from the existi:^ teacher staff if qualified applicants can be found. 
They should have a xniniinum of five years of teaching experience or 
the equivalent^ have a major or a sj^cialization in reading, and 
have demonstrated their ability to work with other teachers through 
pr evi ous in-service workshops or similar means. It is prefen:^d that 
they have masters degrees j but, that qiialifi cation c-m be waived pro- 
vided related experience is siibstituted* 

Duties*. The resource teachers shall work throughout the school 
day to assist the teachers in one building to individualize reading 
instmiction* V?ith the exception of providing a minimum of once^a- 
month group meetings for that building, the resource teacher shall 
provide the -foUoifing assistance only upon the request of the class- 

^ . ■ >> - d 

room teachers ^ . 

1 . Assist in the selection of bo%s and materials that appeal 
to specific children and are written at their level. 

2. Provide information about and -directions-, for use' of those 
items that were presented in the formal* inservice training 

■ sessions. ^ ' 

3. Provide reg^ar assistance in working out a diagnostic 
teaching plan for an individual or a group. 

U. Demonstrate in the teacher's classroom some of the tech- 
niques and materials that can be used to in^'vidualise 
insti^ctioh for a particular, student. 

Give an individual dia^osis of a problem else as a service 

s 

and aa a demonstration of how to test and diagnose* 



6 . 
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Have available 'for general use a file of materials and 
books appropriate for use hy t^ disadvantaged jxjpulation. 

7. Conduct regular (montlily) group discTissions related to 
individualizing reading. 

8. Qive an appraissl of teaching, especially as it relates to 
Ij^dividualizing reading Instnaction. 

Materials Component 

Each of idle - pro jetd;' teachers will be alloted a sum of money 
to purchase books and materials idiat meet the indiyidtial reading 
needs of the c h i ldr en in his class. The original recoimnendation of 
$2,000 per classroom must be modified according to the ovei*all budget 
of the project. A miniiiium of $1,000 per classroom must be 
available. Teachers will selects the— materials on the basis of knowledge' 
criteria gleaned from the formal in-service training programs 
and from the personal recommendations of the resource teacher in the 



building, These purchase 3 should make it possible for each child in 
the school to read at least five different books right from his class- 
room library. 



All requisitions for these materials must be signed by the 
Building resource teacher and processed throu^ the office of the 
Director of Federal I^ograms. 

Budget 



28 substitute teachers for ^ full days + 8 one-half days 
© $20/day 

U resource teachers @ $10,000 * << 

2 consultants for 5 full days + 8 one-half days © $ 300 /day 
materials / 



$ 5,oUo 

ii0,000 



5,U00 



Total - $100,000 
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KSSBOHSE SHEET 



^ seems as if the new Title Z reading project is ready to be sent 
' to the State Department for. review.^ A series'' of general program and 
specific performance objectives has feen prepared, and an operating 
■ plan is read^. In looking over the plan. Vial realizes that it is 
incomplete - v 

yJ From the point of view of an evaluator, %hat should be added to 
,the plan in terms of objectives, in^xLementation, or econorngr? 



I; ' 



objectives 



implemeiitetion 



budget 



•, \. 



/ At what; point in the CIPP Program 
are 'ypu now? -v; ... 
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0I4NISCIENT COMIiENT 

, Having seen the list refined objectives and the program plan^ 
you may feel overwhelmed at the thought of 'trying "to detennine what 
needs "to be added. It would be worttaThile, therefore, to review -the 
manner in which an evaluator assesses a program plan as it develops. 

The program plan — the objecti'VBS and ■ttie proced^Ires of operation — 
is a fulcrum on which program evaluation balances. The plan reflects 
what -was learned about, the needs and "the capabilities of "the school 
system, and it sets up the beha'viors and -the machinery 'upon which the 
judgments of success and failure ■will be made. As an evaluator, 
■therefore , you should spend considerable -time and effort on this 
phase of -the en'berprise. 

The x>rogram plan began when the curriculum committee decided that 
indi'vidualizing reading ins'bi^ction would be accomplished best through 
emphasis on training teachers. At that point, your concern was ■the 
question: ■oror^ram plan Xfill best acconplish the overall goal? 

The answE one., al'tematives and a list of priorities based 

upon research. | 

La'ber you took on the job of refining the commit'tee's objecti'ves. 
Tou refined, expanded, tod resta'ted them in ■terras of specific behaviors. 

You had, to consider again "the resources needed ■to apcoiiplish the 
behavioral' objectives and whether ■these resources were already available 
in the school system or if they had "bo be purchased. Were the books 
and the facilities ready for increased enpliasis on individualization of 
instruction? What stra-tegies could be used to gain acceptance 

At what point, in the CIPP ft*ogr^ 
are you now? . -Input : planning details 



67 



of the objectives? Are the teadiears and administrators willing and 
able to cany out a plan ■ to increase individualization of leading' 
instruction? ShoTold some pilot study be run to acquaint the faculty 
with the idea before it becomes an operating entity? 

-I ■» 

The curriculttm' committee set up the basic framework of an ingjlemen- 
tation design. It said that the Title I program would consist of in- 
service meetings, resource teachers to provide continui ng help, and 
materials for immediate use in the classroom. A special task force 
is assigned -to i^rite implementation procedures. As an evaluator, you 
assess the planned procedures on the basis of their relevancy, feasibility, 
and econiOTy. You also expect the inpleraentation plan to describe a 
plan for monitoring the ongoing program for there are many contingency- 
decisions that may arise about what works for teaching -the objectives 
of the “pj^ogram. iBJhat are the predicted questions that you will have 
to answer about the program? Will you be prepared to answer -those 
cpies-tions on the basis of -the psrogram plan that has been submit-ted? 

Thus, at -the program planning, stage , the con-ten-t- with w^c^h "the 
evaluator works is objectives, resources, stra-tegiea, an irapLement’ation 
cJesign, aid a monitoring system. The criteria that you apply to that 
content are crelevanc^, feasibility, and econoiry. A chart might guide you 
in this phase of your work. " 



■■ ^ ■ ■ ‘ — r — ■ ^ 

In-bcntion: Individualize instruction through focus on teacher. 




Objectives 


Procedures 


Relevancy 


Feasibility 


Econony 


1. 

2. 










3. 












- 
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view of the intention of the program^ check off the matched 
objectives and pTOcedures against the three criteria. 

Relevancy . Is it designed to prodnce^a behavior that will promote the 
individualizing of reading instruction? Is there evidence available? 
Feasib'^T'^ty . Do vre have receptive personnel aj^ adequate facilities? 

Can we adjust to carry out the pregram? 

Econony . Can the same goal be accon^lished through another means in- 
volving less 63Qjense? Can vre afford to do it well? 

In order to answer those questions, you' im^t tore to research 
evidence, cons^^lt the- practitioners and the authorities, and examine 
critically the facilities and the maiterials involved. 

In other words, yolr evaluation procedures would include turning 
to information resources on reading research, such as ERIC/CRIER, 
the. clearin^ouse on research in reading j calling on authorities, such 
as your' consultants or those who have written extensirely about 
individualizing reading; surveying the teachers to get their reactions 
to the objectives and proposed procedures; settxng up a pilot pregr^ 
to see what has to adjusted when the money becomes available- and ; 

the full program begins; and, of course, applying logic and common sense. 

Since it is practically in5>°ssible for us t ' all of|. those 

steps 5 n these simulation training materials, let us assume that Val 
looked .at these aspects of the program plan and applied the criteria. 

.let us further assume that he was generally satisfied with most of the 
in5)leinentation plan; but he f^led to find a monitoring s^tem in it. 
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Deceriier 
Before Chris'tanas 

^ I have revj.ewed the program plan submitted by the task force. 
Generally it looks like a workable program. 

-One major fault that I see is that the task force did not suggest 
azjy plan for monitoring the implementation of the pro^ara or, for 
appi\aising the program at the end of the year. In view of the tight 
schedule we- have, I vjill draw , up a moni'Jjoring plan and submit it to 
the task force. Tliat should enable the task force to put objectives 
and implementation' together for a project proposal before the January 
deadline . 

\/jL- 



GIZ SCHOOL SYSTEM 
Interdeparfenental' Communication 

TO: Dr, R. Leader DATE; 

FROH; Val U. Ator 

SUEJSCT: Program Plan for Title. I 



RESPOIBE SHEET 



-What kind of strategy woiild you- use in setting up a monitoring system for 
the program? . Iferhaps you could formalats a diagram that t«iU assist you 
in knot^fig idiat data “to coULeot^^ as we^ as knot^rg how and when to 
collect it, and hoTT you will do it. 



r- 




At what point in the CIEP Epogram 
are you now? - '/ - 'V' 
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(»1NISCIE3JT COMMENT 

t 



A Bionx'boztliig system should be bused on the anticipated decisions 
that can be made about the projectp The ability to^juake decisions 
depends on the flexibility of a prograiiij the objectives of -liie program, 
recycling possibilities (both immediate and long range). For 
example j should other teachers parti cdpat© in a board-financed 
in-service program next year? A-monitoii.ng system must be planned to 
collect data regarding the oa-gc g activities as well- as the ultimate 
iu^jact of the- project. The monitoring system xaust examine the con- 
gnience between stated objectives and procedxures and the observed 
growth toward objectives -as well as the perfonnance of procedures* 

ThA materials were found in all classrooms. Finally, the monitoring 
system must be designed to evaluate the effectiveness of the particular 
program «xat has been planned. This is often referred to as con- 
tingency evaluation. For exampl.e, the Suf)eid.ntenidert of Giz School 

I 

and' certainly t^te Congress of the United States is interested in an 
evaluation of whether or not this particular program is meeting the 
broad goal of Title I of ESEA. 

How a monitoring system be planned to consider all these concerns? 
A first step ,is to identify those people ^concerred with t>be project- 
who have decision-making power. Seqond, it is necessary to delineate 
as nesdly as possible the decisions that can and need -to be made and 
to plan measurement and 'observation schedules which will provide these 
decision makers with timely, reliable,, and valid information that they 



need to make decisions. 





At what' point in -tee CIPP Program — _ ' ' 

ars you now? Plann ing a monitoring system is part of Input Eval^txon ; 

"carryt^g out the monitoring is part of process ey^uation . 



-San?)le headings for these steps are gi'vren below s 
Deciglcan Malcer Types of Decdsioa " Data CoHe.ction , 

' The chart on the following pages is an attenrot to outline a possible 
etructure for organizing a monitoring system for this project. The chart 
is not meant' to be conprehensive, nor do we believe the only structmre an 
evaluator might utilize. • Any structure shoUl.d^ however, include some 
treatment of the major items on this outline. . ^ 

The first column of the outline. Level of Decision , does not mean 
that one r^cislon maker is Piore- ln?»ortant), than another, but rather it' 
indicates that some decision makers are Interested In a project on a micro- 
level and others on a macro-level. 

The second column. Decisions , is one that is always undergoing 
expansion and modificationi however, the evaluator must be able to 
anticipate as many of these decisions as possible. If he does not, he * 
will be unable to provide the decision makers with vali^ information when 
they need it. These decision statements are determined f^om di^ussions 
with the project personnel and from an exaini^tion of the preset plan. 

The third column indicates wfaether the decision is concerned with 
congruence between wap planned &iid wnat occUI-ja.or the 

contingency of what could have been achieved and idiat was adlaieved. 

Most CEff the SmTusdiate decisions of the project are concemeS 'with con- 
graen^es wtiile long-range decisions are concerned with coisMfi^encies. . 

TBie fourth and fifth coljurans are examples of kind^' of data 
and information that might be collected and of possible me^ur^s that 
might be vised In order to collect these data and provide a limsis for 



maTrfTig- decisions. The final colmro indicates the time by -Hhen the. 
decision” will need to be made. This factcar is extrenBly dn^ortant for 
the evaluator to determine if be is to plan a schedule for 'data 
collecting that will provide information when it is needed. In the 
outline tdxich follows ^ "continuoxisly" is written in iiie Time of 
decision column for most decisions. In reality^ this would need to. 
be moi*e specific; even if decisions are to be continually considered, 
a schedule for collection of data should be made. 



Schema for Project Monitoring Systera 
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Resource teachers changes are needed X X Effectiveness ' Attitude scale Gohtlnuwsly 

within the in-service ' ^of ;^e in- of structured 

program? service program' , visitation ' ^ 

. ' in generating 

V, •■ • ' change • - • 
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Several months have passed. The project proposal on individiializing 
reading instruction went to the state office for^ Title I and was approved. 
You have been busy in the intervening months trying to get everyone 
ready for the start of the program. You have also been working on the 
.projects for which you-.are personally responsible s. You have been 
developing instriiinents for data collection, validating them, and 
instructing your assistants in how to use them. You have worked veiy 
hard at developing a reporting network. Various teachers and adminis- 
trators have agreed to help you in the monitoring and data collection 
process. You have also xnade yourself moi*e familiar with the decision 
makers and their idiosyncrasies. You must tailor each of your reports 
to the respective decision maker. 

. But the program, is actually functibiring now. You have to see 
whether al l your planning Xfd.ll help in the efficient operation of the 
program and in fighting the fires that sparing up. 
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As the new Title I reading project sxTings into operation, Val is 
busy with implementing the project monitoring system* This includes the 
develofHoent of a time table on which Val has listed decisions that are 
to.be made, vtio is to make them, when the decisions are to be made, and 
procediires for collecting the data. In addition to decisions to be 
made, Val has listed a number of immediate and long-range objectives 
'idiich aare to be assessed dux*ing the project. Fi n a li ty , Val has listed 
on his master plan points at which he is collecting data- on the 
roachanical operation of the project. 

To iit^jlement this design, Val continues to select and develop, data 
gathering proceduresi he meets with project personnel to discuss short- 
term objectives and to interpret data to various project people so 
"decisions can be made V 

Val bag been gathering data on various random groups of teachers 

to determine if the- in-serv4ce-activi4ie&-are-having— imme^ate-effects - 

on classroom practice. He has been inforeiing Mrs, Ima Goodteacher, one 

of "Hae resource teaciiers, of the results of these activities and is 

' s 

therefore somewhat surprised when he receives the following memo. 
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GIZ SCHOOL S2STEM 
Interdepartmental Connruni cation 



TOs Val DATES Hallaxreen 

Xear j'/2 

FRCMs Resoxorce Teacher 

SUBJECTS Reading Program 

Val— 

I would HTfa to arrange a conference with you as soon as 
possible. Ibis is the second month of my classroom observations, 
and I find that several teachers are not attempting to change their 
program. This group seems to be led by Mrs. Doga Matic. She seems 
to feel that children are not capable of handling an individualxzed 
program and that mbi^ control is needed in the classroom. She sees 



Tibthihg ■wrong'TH.tti wnat she has^ been doing in the past. 

I am afraid her influence may spread. I would like to talk 

. i 

with you before attaching the problem. 









Ima Good teacher 
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EESIOiiSE SllEET 

As the project evaluator, wha4 plan should you have made at l^e 
beginning of -ttie inject to answer Ima*s meioo? Obviously she has a 
problem. She has several alternatives, including not trying to 
influence Doga Matic. If you had not planned any data collecting at 
the beginning of the project in regard to this problem, what could you 
do now? 




\ 



At what point in the Clip I^ogram 

; ■ are . yDU''now?-.- :/ 'v. •'■"■r;./-.-. : ■ 
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OMNISCJENT COHMSNT 
Fal*s Wbrifing Notes 

1 . I should help lina to see the problem as clearly as possible, 

interpretation is that she is concerned because one (or 
more) of the teachers are not changing their cslassroom behaviors. 

2. I will talk to her about liie da^sa ihave been collecting on 
various random sai!g>les of tea<iiess, 3 ies© tindicsete 'Qiat many 
teachers are making significant c^ias^^s* 

3« We should consider the possibility e£f more ^systematic data 
collecting on tiiose teachers lina lihiaifcs ais imost resistant 
to change. We should go cautiously iSEJ thi^„ however. 

I4., r will discuss with her the result® of same of the ’•cognitive 
knowledge tests," These indicate "that almost all the teacJxer.s 
understand the ideas in the in-service project, X will also 
check s]^cifically on Mrs. Matic’s results. 

5, I will discuss with Ima several possible alternative strategies 
for working with Mrs. Matic. lina probably has several ideas 
fll npg these lines 4 and we can examine the strengths and weaknesses 
of each. 



At what point in the CIPP Program 
are you now? Process evaluation 
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Shoriily after Thanksgiving (Tear ^2), Val is confronted wiiJi anotiher 
evaluation crisis* A memo arrives from H. leader* 



GIZ SCHOOL SYSTEM 
Interdepartiosntal Coinraunicati&u 

TOs Val U. Ator Tlianksgiving 

Tear #2 

FROM* R* Leader 

- SUBJECT* Letter from Mrs. Brow 

Val— 

T3ie attached letter was sent to me from Crabtree over at 
Benton* Mrs* Brow, sounds pretty upset and I*d like to do 
something before this turns into something big* I*m con~ ^ 
sidering sending out a memo to the parents of aU. the 
children whose teachers are taking part in this program. 

Can you give mo some information on this? Has your evalua- 
tion been keeping tabs on the effect of teachers’ absence 
oh students’ performance? Tour inxtial studies didn’t 
include any information about parents > did they? let me 
know what you come up with* 
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8’3 



Mr. Crabtree, Principal 
Benton School 



Dear Mr. Crabtree, 

I tmderstand from my son. Fairies, that his teacher -will not be 
in school, all of next w^eelc. I sinceisly hope she isn*t ill and am 
going on-the asstni5)tion that this is not the case since this seems to 
be a planned absence. Fairlee has be?an maiding wonderful progress -^tte 

Miss Gone over the last few weeks, amd I feel certain that 
marking tTnw xjith a substitute teacher will set him back so far that xie 
will be unable to catch up and master second grade work ^ the end of 
school* I have spoken *bo a number of the other imo there in Miss Gone s 
room, and you can be assured I am not alone in my concern. I am 
hoping you will tell me how you can justify excusing a teacher from 
her duties for a xdiole week. Please keep in mind that not oniy our 
children *s futxjres but our tax dollars are at stake. 

Sincerely, ’ 














Bk. 



RSSS0H5E; SHEET 



space is for ytnxr trorldng notes to mspond -to R. Xei;^f*fer,*s 
mtsuto* VJhat kinds of informaticwi wou!W 3PU galiter to help Iead»« r m a ks 
^ 'ieclsion? . 



A 











■ ■ ■ ; :■ 

At what point in the CII^ Ibrogram 
are yoix now? '' " • V '-v^' ' 
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(MUSCIENT CQMMESMT 
' Val‘a Notes 



1 . Discuss 7 jith leader his interpiratation of iiie, memo in order 
to get a better miderstandingi of the problem. 

2« Determine how soon lea der feels he needs to act. 

3 , Suggest several facts he mi^t consider bef or*e acting) 

' ^'"' and have plans ready for telling leader how these can be 
done quickly. Pbs_sibilities include— 

a. Detenninang parent knowledge of the program through 

a telephone interview. A random sajigjle of parents could 

19 

be used. 

b. Collect information on the qualifications of the substitutes. 

c. Monitor classroom activities on the days teachers ai*e 
absent for in— service program. 

d. -Go31ect-ihforniation-con5)aring-in-services-program--teacher^s^ 

absences with the average absence rate, 
h. Work with Leader to specify alternative plans to meeting the 
problem and help him to assess these© Consider criteria for 
selecting alternative to be followed. 




\ - O . 

\ ■ . - ■ ■ - - . ■ . ' ' 

\ . ■ ■ 

\ 



At whaKpoint in the CIPP Program , 

are you now? Process i contingency declslg:^ 
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Anothar memo arrives ftom R. leader before the C2irlstinas holidays. 



GIZ SCHOOL SYSTEM 
Interdepartmental Conmruni cation 



TOs Val U. Ator BATES 

5ROMs R. Leadei:' 

SUBJECT: Pregnant Lad ies 



Val~ 

One of the fovir resource teachers has just annoimced 
that she will be resigrring at the end of next month due to 
pregnancy. X^hat information is available to help us de- 
cide whether to hire another resoiarce teacher or to spend' 

( 

the extra money on materials? 

This is a good year— only one pregnancy so far! the 
way, how are your ulcers? 



Christmas 
Year //2 






O 

ERIC 
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HESK)HSE sIeET 

V 

Hew Hotild yt>n x^sponA to Xeadl0r*s loeino? 




{ 




■ • • • t 



At iHbat point in the GIFP B^ogram 
^aro-you.-n 




OffTDSCJlENT COMMENT 
Q Vial’s Rotes 



1. Ihe isroblem is dear and the alternatives have been 
d e li n eated* The job is to gather infomation to assess 
the altematiWs* 

2. Tour response to leader ni^t go something like the follcwings 



GIZ SCHOOt S25TEM 
Interdepairianentsl ■Communication 



DiTSs Christmas 



TOr - Dr.,R* leader 
FR^s Val U. Ator 

SUBJECTS Resource Teachers 

Concerning the question as- to sdiether to hire another . resource 
teacher or use the money to“ buy materials , I have reviewed the 

p , 

teachers* monthly reports azid have gleaned £roia 12xein their 
reactions to the— resource teachers* I have also surveyed the 
resource teadhers’ log boc^ which contain , a listing of ' their 
activities and requests for assistance. had -Uie principals ask" 
for the teachers' ccmsaents on their desire f^ \nore materials* 

The infon^tion .X have coUected is thiss 

1) Most teadbers feel that the iresource teachers are contributing 

to the new program* - ^ 

2) .Mary teachers are requesting indivldxial help sessions or deraon- 
si^atlon leajsons* All resource teachers have a busy schedule* 

3) There have been -few comments about the lack or inadequaiy . of 

materials* Teachers seem i^o feel they need to e;£plore the 
possibilities of the ^materials dready acquired* - 



P*S» Ify deers are relatively, calm* I took your suggestion about 
; eating yogurt* 



At what point in CI^ 

are you now? vRrocesiB *: ^cbhpgrondy decision 
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A ineiad from ivhe A-T Directior# 



69 
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RESPONSE SHEET 



Respond to the A. V. Director, What are reasons for trouble, 
and how can you help Ifr. Video decide on hoiT to get things moving? . 



I 

r 

■ i 

1 




At what ppi 
are you now? 



the GII^ Program 






r> 
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VaT Possible Re^onse 

/V’ 

XOs Sal Video /’ 

raOMs Val tJ. A-fcor j i 

I axs aware of the pu^oblem conceicing "the A-V materials* I have 
reviewed the resource teacher’s monthly ' reports on this topic> the A^V 
circulatioh log, and 1 have sangjled teachers’ opinions by interviewing* 

I have isolated th^'''foUowing possible reasons for the lack of use 
of the new equipment! 

1 • Teachers feel uneasy about the actual mechanicax operation of 
sons of the equipment. The ways in which the equipment may be 
used has been demonstrated, but there hasn't been much practice 
with 'Qte nschanics. 

2. Teachers find it inconvenient to visit the material center to 
diecik out equipment* Some teachers have never been to the nevr 

, center^ and a few do not know where it is located* 

I would suggest these possible alternatives ! 

I, ,;.Have a materials center open house. ^ ^ ^ 

2* Develop a more efficient method af ordering and distributing the 

equipment* ? - 

3 , Plan a short session” where teachers. -actually practice using the 

eqUil^ment* ' ' 

At what point in the CIPP Program 

are you now? Process evaluation! management of laroc edures 

9;4; ^ " ; 




An educational evaluator often magr be called upon to provide help 
in "crisis'’ decisions — soma minor, some quite important. ’She important 
point is that if he has the monitoring machineiy operating, he can 
assist the decision makers in modifiring the program where the evidence 
shows a need to change objectives or procedures. For exan?)^, if 
reports from the resource teachers indicated that the classroom teachers 
were not calling upon them for observing teaching and offering suggestions 
for in5)rovement, it might he necessary to modify the operatioi^al guide- 
line that said the resource teacher woidjd observe in the classrom only 
lahen asked the cXassroom 'teacher* \ 
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TIxe first 3rear of the Ktle I reading program progressed without 
too much trouble. Val had other occasions where a decision maker 
needed data to solve a crisis. But his ulcer calmed down along the 
tray and that seemed to s^ something for the project. Val knew that 
at the year's end he had to prepare some summary statements about 
the effectiveness and worth of the program. He looked at his monitoring 
plan to see >jho the major decision makers were and began jotting notes 
on the kind of final report he woxild need to vjrite. 
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HESKJNSE SHEET 

From vheA you know of the Giz School System and its Title I leading 
program^ can you determine the consumer grot^s of final repo3?t informa- 
tion nnH the kinds of reports they would need? 3h other words, how 
many different reports will you have to prepare in order to meet the 
needs of the decdsion-roaker consumer? 

Decision-nakizig Brief d®acrlg)tion of report aimed 

consumer at a spedficc cansumer's needs 








j V . ; , At what point in the CIEP Trqgcsxn 
\ ' are';-you :.hbw?:: 




aaCESGnElJT CC231E3fT 

t 

Host evaluators will find it isecessary to prepare several "product'’ or 
"stunniative” reports. Bie following ai*e e-sainplss of the 3dbv3s of itepoi*ts 
you msy have listed as part of your working notes. 



DecisionHinafcing “ Brief descriptiion of report aimed 

consuDBar a.t a specific coxisumer's needs 

< 


Dr. R. Leader 


1. Analysis of prohlesjn area 

2 . Future plans revhs^oais 

3, Altematiwas of ingiruve- 

" — — — ment and budget rasEision 

recommendad 


Board of Edacation 

% 


Progress report: SunBatry of year’s 
activities and "State df Union" 
report. J 


Task Force Coimnittee 
Curriculum C^nmittee 
In-service Comsidttee 
State Title I Office^j^....^-'^'^ 


1 . ' Evaltiation of objectives 

2. Problem areas 

^ 3» Equii»nent and raateiials, 

inventories and equipment-- 
usage charts 

U. Student progress reports 

T. Statistical report 
2. Harrative report 
3» Evaltiation 


TJ. S. 0. E. 


% 

\ Forra^. report of activities, students 

involved 

\ 



At what point in the CIP!P ft'ogram 
are you now? l*roduct avalixation 



